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Abstract. Psychophysical experiments have demonstrated that it is possible to perceive both 
binocular depth and rivalry in plaids (Buckthought and Wilson 2007, Vision Research 47 2543-2556). 
In a recent study, we investigated the neural substrates for depth and rivalry processing with these 
plaid patterns, when either a depth or rivalry task was performed (Buckthought and Mendola 2011, 
Journal of Vision 11 1-15). However, the extent to which perception of the two stimulus aspects was 
truly simultaneous remained somewhat unclear. In the present study, we introduced a new task in 
which subjects were instructed to perform both depth and rivalry tasks concurrently. Subjects were 
clearly able to perform both tasks at the same time, but with a modest, symmetric drop in performance 
when compared to either task carried out alone. Subjects were also able to raise performance levels 
for either task by performing it with a higher priority, with a decline in performance for the other task. 
The symmetric declines in performance are consistent with the interpretation that the two tasks are 
equally demanding of attention (Braun and Julesz 1998, Perception & Psychophysics 60 1 -23). The 
results demonstrate the impressive combination of binocular features that supports coincident depth 
and rivalry in surface perception, within the constraints of presumed orientation and spatial frequency 
channels. 
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1 Introduction 




The possibility that stereoscopic depth may be perceived in the presence of rivalry has been 
a longstanding controversy in the field of binocular vision. A number of studies have shown 
that depth and rivalry interfere with one another, and based upon predictions of models 
of binocular vision we would not usually expect that they can be perceived simultaneously 
(Blake 1989, 2001; Harrad et al 1994; Hayashi et al 2004). In particular, the assumption is 
often made that depth cannot be perceived in the presence of rivalry because inhibition of 
one stereo half-image would prevent a match to the other image (Harrad et al 1994; Hayashi 
etal2004). 

However, it has also been proposed that distinct spatial frequency channels are involved 
in binocular depth and rivalry (Julesz and Miller 1975). Moreover, it is actually possible to 
perceive depth in the presence of rivalry, as long as the components are in different spatial 
frequency bands (Blake et al 1991). Another study showed recently that it is possible to 
perceive both depth and rivalry in one spatial location (Buckthought and Wilson 2007). This 
was observed with binocular plaid patterns in which depth is perceived from the near-vertical 
components, and rivalry from the oblique components. Moreover, this may be observed 
even when the depth and rivalry components are at the same spatial frequency, as long 
as these components are in different orientation bands. The percept of a rivalrous pattern 
appears to be spatially superimposed on the tilted surface. Recently we investigated the 
neural substrates for depth and rivalry processing with these plaid patterns, while a depth 
or rivalry task was performed (Buckthought and Mendola 201 1). We documented a similar 
pattern of activation across cortical areas for both tasks, with some modest differences 
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between the two. Nevertheless, it was not clear to what extent perception of depth and rivalry 
was truly simultaneous, since subjects were never asked to report both percepts at the same 
time. One recent psychophysical study reported that it is possible to perceive depth and 
rivalry simultaneously; however, depth perception with the stimuli used in this study likely 
reflected second-order processing (Andrews and Holmes 2011). Hence, it still remains an 
open question whether it is possible to perceive depth and rivalry simultaneously through 
first- order processing. This is a question of considerable theoretical interest worth revisiting, 
since current models of binocular vision would not predict that this would be possible (eg, 
Hayashietal2004). 

Our work on the coexistence of depth and rivalry fits within a body of literature suggesting 
that binocular integration and rivalry can occur independently for different stimulus features, 
including colour, form, and motion (Andrews and Blakemore 1999, 2002; Carney et al 1987; 
Holmes et al 2006; Hong and Shevell 2009). For example, chromatic integration can occur 
independently of form rivalry (Holmes et al 2006). Specifically, rival stimuli differing in form 
and colour can sometimes achieve states of dominance in which the colour information from 
one eye's image combines with the form of the other eye's image (Holmes et al 2006; Hong 
and Shevell 2009). In a second example showing separability of motion and colour, the colour 
information from one eye's image may be suppressed, while the information necessary for 
motion perception is integrated (Carney et al 1987). In a third example showing separability of 
form and motion, moving gratings of different orientations were shown to the two eyes, and 
motion perception was studied during time intervals that one grating was suppressed. Pattern 
motion was still perceived, indicating that the motion component signals were integrated 
across the two eyes and the perceptually suppressed grating influenced the perceived motion 
(Andrews and Blakemore 1999, 2002; see also van Boxtel et al 2008). Collectively, these studies 
provided evidence that the neural processes involved in integrating information from the 
two eyes can act selectively on different stimulus features. 

In the current study, we carried out psychophysical experiments in order to determine 
whether it is possible for subjects to report depth and rivalry percepts concurrently with 
plaid patterns. For the sake of comparison, we also tested performance for grating stimuli 
that only contained depth or rivalry. We were also interested in determining whether these 
two tasks differ in attentional demand, as this would be relevant to the interpretation of both 
psychophysical and fMRI results with these plaids. We relied on methodology used previously 
in concurrent task paradigms (Braun and Julesz 1998). We asked subjects to perform five 
different tasks with the same plaid stimuli: (1) depth alone; (2) rivalry alone; (3) both depth 
and rivalry tasks with equal priority; (4) both tasks, but depth had a higher priority than 
rivalry; (5) both tasks, but rivalry had a higher priority than depth. We assumed that during 
the simultaneous task conditions (3-5) a drop in performance would be evident in making 
more wrong or missing responses for depth, or making fewer key presses for rivalry. We 
compared the performance when both tasks were carried out to performance with each of 
the single tasks. If depth and rivalry have similar attentional demands, then the performance 
levels for these tasks in the simultaneous condition (3) should be similar. 

2 Methods 

2.1 Subjects 

An author (AB) and five other subjects, who were naive as to the hypotheses of the study, 
participated. The subjects (which included four women) were students or postdoctoral 
fellows (mean age 29 years). All were right-handed and had normal or corrected-to-normal 
acuity and stereoacuity thresholds better than 30 s arc, measured using the Titmus stereo test 
(Stereo Optical Co, Chicago, IL) . The subjects provided written consent and were remunerated 
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for their time. The experiments were carried out in accordance with the World Medical 
Association Helsinki Declaration (October, 2008) and approved by the Research Ethics Board 
(REB) of McGill University (Protocol NEU-08-03). 

2.2 Stimuli 

Plaids in which it is possible to perceive both depth and rivalry were produced by adding 
together near-vertical (±2.5 or ±3.5 deg) and oblique (±30 deg) sinusoidal 6.4 cpd grating 
components, at 80% Michelson contrast (Figure 1). Similar gratings that contained only 
depth or rivalry were also used for comparison. 

2.3 Display 

Stimuli were presented on a MacBook Pro Laptop (Intel Core 2 Duo) Macintosh computer 
with 1024 X 768 resolution, 120 Hz refresh rate with 8 bit/pixel greyscale, which was gamma- 
corrected using a colour look-up table. Stimuli were displayed using Matlab and Psychtoolbox 
Version 3 (PTB-3) software. A Matrox (Dual Head 2Go Analogue Edition) splitter graphics 
card was used to create two channels for dichoptic displays. Dual LCD (InFocus LP 540) 
projectors and linear polarizers were used for dichoptic projection (Thompson et al 2008). 
The subjects wore linear polarizers with complementary polarization on their eyepieces. The 
stimuli were back-projected from the LCD projectors onto a polarization preserving screen 
at a viewing distance of 134 cm. Each stimulus was projected through an opaque circular 
aperture (3.8 deg diameter), which minimized any edge disparities. Viewed through the 
polarizers, the stimulus had a mean luminance of 30 cd/ m 2 and peak luminance of 60 cd/m 2 . 
The same display apparatus and viewing distance was used as had been used previously for 
fMRI experiments (Buckthought and Mendola 2011). 




Figure 1. Stimulus patterns, (a) Depth alone and (b) rivalry alone, (c) These components were added 
to produce plaids in which both depth and rivalry may be perceived simultaneously. 
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2.4 Psychophysical tests 

2.4.1 Rivalry task 

Alternation rates were measured during binocular rivalry with the plaids or, for the sake of 
comparison, with left/right oblique grating stimuli. Subjects reported perceptual alternations 
continually over each trial. Subjects pressed one key when the left image predominated (over 
at least two-thirds of the stimulus) and another key when the other image predominated, 
using the same criterion. Subjects were tested a second time with the meaning of the keys 
reversed. 

2.4.2 Depth task 

For the depth task, subjects reported whether the top or bottom of the plaid stimulus pattern 
appeared to be tilted forward (or for the sake of comparison, with depth alone grating stimuli). 
The orientation disparity was 5 deg. The depth changed every 3 s (this interval was chosen 
based on pilot studies, so that it was not significantly different from the mean time interval 
between key presses in the rivalry task, averaged across subjects). In order to prevent the task 
from being too easy, there was one catch trial chosen randomly at every 20 s interval in which 
the disparity did not switch to the opposite percept but was changed to a slightly different 
orientation disparity (7 deg). Subjects were not told how many catch trials there were. 

2.4.3 Concurrent task conditions 

Subjects performed five different task conditions with identical plaid images in which there 
was both rivalry and depth changes every 3 s: (1) depth alone; (2) rivalry alone; (3) both 
depth and rivalry tasks with equal priority; (4) both tasks, but depth had a higher priority 
than rivalry; and (5) both tasks, but rivalry had a higher priority than depth. Subjects made 
responses for the two tasks using four different keys, two keys for rivalry appearance and two 
other keys for depth appearance. The length of each trial was 90 s. Subjects were instructed 
to make key presses only when they were able to clearly perceive a rivalry alternation, or 
a depth percept. Subjects were given a practice set of trials, then tested four times at each 
condition. For the depth task, we compared the performance in terms of percentage correct 
under the different task conditions, while for rivalry we used alternation rates. The order of 
the psychophysical testing for all of the different task conditions was counterbalanced to 
avoid the effects of practice or learning. 

One additional control test was subsequently used. After the main testing sessions, three 
subjects (AB, LA, JH) were also tested a second time in versions of these tests in which the 
time interval between switches in depth was jittered randomly, so that the interval varied 
unpredictably, but with a mean of 3 s (the following time intervals were used with equal 
probability: 1.5, 2, 2.5, 3, 3.5, 4, 4.5 s). This was a better match to the temporal dynamics of 
rivalry, and also made guessing more difficult. 

3 Results 

Our first comparison between the rivalry rates for the plaid and sinusoidal grating patterns 
(when the rivalry task alone was performed), shows that mean rivalry rates are decreased by 
the addition of depth cues (fdf= 5 =2.77, p = 0.039). There was also a significant difference in 
the distribution of time intervals between key presses (Kolmogorov-Smirnov Z = 2.49, p = 
0.013). This confirms results from our previous experiments that there is some interference at 
the perceptual level resulting from the addition of the depth cue in the plaids (Buckthought 
and Mendola 201 1; see Buckthought and Wilson 2007). 
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Figure 2. Depth perception judgments (% correct) and alternation rates for binocular rivalry for all six 
subjects. Performance is shown for the single task and concurrent task conditions, in which the task 
had high, low or equal priority. Subjects were able to perform both the depth and rivalry tasks in the 
concurrent conditions, although performance was lower than the single task conditions. Error bars 
show ±1 standard errors. 

In our next manipulation, we compared performance on the five task conditions for 
identical plaid stimuli in which both rivalry and changing depth cues were present. Depth 
perception and rivalry alternation rates are shown for all six subjects in Figure 2. The 
performance is shown for either depth or rivalry in the single task, or for the concurrent 
task conditions when it had the high, low or equal priority. Although absolute levels of 
performance could be quite different, all subjects showed a similar pattern of results when 
comparing performance on the single task relative to the other task conditions. In particular, 
the baseline rate of rivalry differed across subjects (Figure 2, left-sided y-axis). Nevertheless, 
all of the subjects were able to perform both the depth and rivalry tasks under all of the 
different concurrent task conditions, lending support to the hypothesis that these percepts 
were indeed simultaneous. For example, in the simultaneous condition when the two tasks 
had equal priority all subjects were able to perform the depth task, since performance levels 
ranged from 74.1% to 84.8% and were above chance for all subjects (chance performance is 
50%). Nevertheless, it can be seen for each subject that the performance decreased somewhat 
in all of the concurrent conditions compared with the single task conditions. 

In Figure 3a, the individual results have been averaged across subjects and show the 
same trends. In Figure 3b, the performance for the concurrent task conditions have been 
normalized relative to the single task condition, where a value of one would indicate equal 
performance (again, averaged across all subjects). Overall, the similar results for depth and 
rivalry indicated that the attentional demand did not differ for the two tasks. There was 
clearly a performance deficit for either depth or rivalry during both the low and high priority 
simultaneous task conditions, relative to the condition in which either task was performed 
alone. But interestingly, this symmetrical performance deficit was always similar for either 
depth or rivalry, implying that they required equal attention. 

Statistical analyses were carried out to confirm these trends. A two-way repeated measures 
ANOVA was significant for the differences between the single and simultaneous task 
conditions with equal priority for the two tasks [Fi$ = 44.4, p = 0.001) but was not significant 
for depth versus rivalry (F 15 = 1.85, p = 0.23) nor the interaction (Fi j5 = 1.85, p = 0.23). 
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Figure 3. (a) Depth perception judgments (% correct) and alternation rates for binocular rivalry as in 
Figure 2, but averaged across all six subjects. Error bars show±l standard errors, (b) The psychophysical 
data has been normalized to performance in the single task condition, which is assigned a value of 
one (mean of all subjects). Again, subjects were able to perform both the depth and rivalry tasks in 
the concurrent conditions, but there was a symmetric drop in performance for either depth or rivalry. 
Performance for either depth or rivalry did not differ significantly in either the high, low, or equal 
priority conditions. Error bars show ±1 standard errors, (c) Attention operating characteristic (AOC). 
The performance levels from (b) have been plotted as a scatter plot. As performance on one task 
increases, performance on the other decreases. 



Crucially, the lack of significant difference between the performance for either depth or rivalry 
in the simultaneous condition suggests the two tasks are comparable in attentional demand 
(?5 = 1.36, p = 0.23). Furthermore, when either the depth or rivalry task was performed with 
higher priority, the performance did not differ (£ 5 = 0.643, p = 0.55). Also, when either the 
depth or rivalry task had lower priority, again the performance levels did not differ (f 5 = 1.25, 
p = 0.27). 

Another interesting feature of the results was that the subjects were able to increase their 
performance on either the depth or rivalry task by giving it higher priority, and this also 
caused a drop in performance for the other task. A two-way repeated measures ANOVA 
on the whole data set was significant for the differences between the five task conditions 
1^3,15 = 50.0, p = 0.001) but not the differences between depth and rivalry (Fj i5 = 0.361, p = 
0.57) nor the interaction (^3,15 = 1.98, p = 0.16). Since the differences between depth and 
rivalry were not significant, the results were averaged together to compare the different 
task conditions. The differences in performance in the high, low, and equal priority task 
conditions were all statistically significant (high versus low: t 5 = 3.67, p = 0.014; high versus 
equal: £ 5 = 3.20, p = 0.024; low versus equal: t 5 = 3.59; p= 0.016). These results are also shown 
in the form of the Attention Operating Characteristic (AOC) (Figure 3c), in which performance 
levels for the three concurrent task conditions are plotted as a scatter plot (Sperling and 
Melchner 1978). These lie on a straight line, indicating that performance on the depth task 
increases while that on the rivalry task decreases and vice versa. The AOC scatter plots for 
individual subjects (Figure 4) show some intersubject variability, but all subjects showed 
the same overall trend. This further confirms that the two tasks did not differ in attentional 
demand and that performing the two tasks concurrently results in symmetrical declines in 
performance (Braun and Julesz 1998; Sperling and Melchner 1978). 

Following the main experiment, three subjects were retested on a control experiment with 
the same five main task conditions with identical plaids, but with the time intervals between 
depth changes jittered. This was done in order to discount the possibility that the subjects 
were only able to perform the original depth task by allocating their attention to the task just 
after depth changes. For comparison, the subjects were also retested on versions in which the 
depth changes were exactly 3 s. A three-way repeated measures ANOVA comparing the two 
data sets was not significant for the effect of jitter (F 1>2 = 1.04, p = 0.42) or differences between 
the depth or rivalry tasks {F\ t2 = 0.271, p = 0.66), but was significant for the differences 
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between the task conditions (F 3 6 = 16.0, p = 0.003). None of the interactions were significant 
(depth/rivalry by jitter, F h2 = 0.160, p = 0.73; depth/rivalry by task, F 3 , 6 = 1.42, p = 0.33; jitter 
by depth/rivalry by task F 3>6 = 0.073, p = 0.97). Since there were no significant differences 
between these two different versions of the tests, subjects were not likely relying on a strategy 
of predictably allocating their attention in the temporal domain. 
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Figure 4. Attention operating characteristic (AOC) as shown in Figure 3, but for all six subjects. The 
performance levels (normalized to performance on the single task) have been plotted as a scatter plot 
for each individual. As performance on one task increases, performance on the other decreases. 



4 Discussion 

Our main finding is that the depth and rivalry tasks could be performed concurrently, which 
is consistent with the interpretation that depth and rivalry are perceived simultaneously. 
This is also in agreement with subjects' subjective reports that the rivalrous percept was 
spatially superimposed upon the tilted pattern. Although these tasks could be performed 
concurrently, there were symmetrical drops in performance for depth and rivalry relative 
to the single task conditions. The drop in performance could be due to a number of factors, 
including a limit on low level binocular visual perceptual encoding, a bottleneck at the 
stimulus selection stage or access to visual short-term memory for these different visual 
perceptual changes, or interference at the motor response stage (Braun and Julesz 1998; 
Klingberg and Roland 1997; Marois et al 2006; Schubert and Szameitat 2003; Wolfe et al 
2006). A number of previous studies of dual-task interference have noted that the bottleneck 
in processing often occurs at the stimulus selection stage (Marois et al 2006; Sigman and 
Dehaene 2008), or at the motor stage, given the difficulty in making motor responses and in 
mapping the responses to the appropriate key (Herath et al 2001). Alternatively, limitations 
in visual short-term memory have been invoked to explain the failure of subjects to identify 
visual perceptual changes which are well above contrast thresholds and difficulty to recall the 
status of recently attended items (eg Wolfe et al 2006). This was studied in paradigms using 
arrays of simple objects in which subjects were asked to report the colour or orientation 
of cued items just after they were removed or masked. The ability to report the features 
correctly was generally poor, but improved if attention had been recently directed to that 
object. Subjects have a very limited ability to detect changes that happen after attention has 
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been directed away from an object and before attention returns to that object (Wolfe et al 
2006). Selective attention in the concurrent task conditions is also related to a mechanism 
by which task- relevant information impacts on ongoing processing, while task-irrelevant 
information is excluded (Roberts and Thiele 2008). 

An important point to consider when interpreting the current data is whether the deficit 
in the concurrent tasks could be simply due to the motor demands of making two sets of 
responses. In considering this issue, we note that previous studies have found that a number 
of different detection tasks are performed in a parallel fashion, so that the required viewing 
time is not prolonged by additional display elements (Braun and Julesz 1998). Generally 
the detection of two sets of stimulus features occurs in parallel when a simple feature (eg, 
luminance, colour, motion, disparity, orientation, size and presence of line terminators) 
distinguishes the target from other display elements (Braun and Julesz 1998). For example, 
Braun and Julesz (1998) found that discrimination of the hues of two targets in the periphery 
entailed little or no attentional cost (ie, the hues of two targets were discriminated and 
reported just as reliably and quickly as the hue of one target) . The colour targets were 
presented simultaneously and subjects were required to make two sets of button press 
responses for the two tasks. The fact that some perceptual tasks can be performed in parallel 
with no apparent deficit in performance due to added motor demands suggests that the 
deficits that we observed here were more likely perceptual than motor. Nevertheless, the 
difficulty in confirming this issue does not detract from our main conclusion that the depth 
and rivalry tasks can be performed concurrently, with approximately equal demands on 
attention. 

The effects of attention have often been compared to the effects of increasing stimulus 
contrast, in that objects or features that are attended to are more salient (Boynton 2009; 
Roberts and Thiele 2008). In neural models, the effects of attention on neuronal responses 
have been characterized in further detail as reflecting a contrast gain, baseline shift or 
multiplicative gain (Boynton 2009). The majority of recent studies of attention can be 
categorised as studies of either spatial attention, feature-based attention, or object-based 
attention (Boynton 2009; Yantis 2008) . In the current study, attention was not cued specifically 
to a particular location in space or feature, so the depth and rivalry tasks may be more 
closely related to object-based attention, as in previous studies in which subjects directed 
attention to transparent surfaces (eg, Ciaramitaro et al 2011). In general, an attention system 
acts on neural representations of the visual perceptual world, enhancing the processing 
of the currently attended object at the expense of information about less relevant objects 
(Yantis 2008). Previous fMRI studies of dual- task interference have proposed that if two tasks 
make use of the same cortical neural resources, then the two tasks cannot be performed 
simultaneously ('cortical field hypothesis'; Roland and Zilles 1998). The assumption is made 
that a particular brain network which is fully engaged in performing one task is not available 
to perform a concurrent task (Herath et al 2001; Klingberg and Roland 1997). 

When considering the current results along with those from recent fMRI experiments, 
a consistent picture emerges — depth and rivalry are processed in parallel spatial and 
orientation channels (Buckthought and Mendola 2011). Both depth and rivalry are perceived 
simultaneously in the binocular plaid patterns because (with the carefully chosen orientation 
differences used here) they are processed in different orientation channels in early visual 
cortex. They may also be segregated by being placed in different spatial frequency channels, 
with spatial frequency differences greater than one octave (eg, 2.5 and 6.4 cpd). The brain 
activation results indicate that early visual areas V1-V3 support high spatial frequencies 
(eg, 6.4 cpd) while at the same time MT+ supports lower spatial frequencies (eg, 2.5 cpd; 
Buckthought and Mendola 2011). Either depth or rivalry could selectively activate V1-V3 
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when it was at the higher spatial frequency, or could activate MT+ when it was the lower 
spatial frequency component. Also, other data suggest that the binocular combination of 
multiple 'cues' — ie depth and rivalry in a coherent binocular percept — selectively recruits an 
area in lateral occipital cortex. In addition to the mechanism in which the correspondence 
problem for depth or rivalry is solved, there may be a representation at the surface-level 
for the grouping of features, in which more than one feature is coded at a spatial location. 
Furthermore, the psychophysical results also help to interpret greater brain activation in 
parietal areas (ie, superior and inferior parietal cortex, including temporoparietal junction 
and intraparietal sulcus) for the depth task than the rivalry task (Buckthought and Mendola 
2011). The psychophysical results rule out the possibility that this could be attributed to 
differences in attention and is more likely explained by differences in binocular perceptual 
processing. 

The results also indicated that these two tasks were not mutually exclusive; the perfor- 
mance levels in the concurrent task conditions were clearly above chance, although they 
were below best performance in the single task condition. Hence each one of the tasks did 
not engage all of the available attention. In general, a task would be expected to have a low 
attentional load in situations in which the task-relevant stimulus is easily detectable, and 
the task-relevant attributes can be processed in parallel at the level of early vision (Braun 
and Julesz 1998). The results were also shown in the form of an AOC, which describes how 
performance on the two tasks varies as a function of the division of attention. This description 
is designed to make inferences about the performance-resource functions (PRFs) of the tasks, 
which describe performance as a fraction of the attention devoted to a task (Braun and 
Julesz 1998; Sperling and Melchner 1978). This description also illustrates that subjects could 
selectivity raise their performance levels for either depth or rivalry by placing higher priority 
on that task. A linear and symmetric AOC is consistent with the underlying PRFs being 
similar for both tasks, and with increments in performance on one task directly related to 
decrements in the other task (Braun and Julesz 1998). This is consistent with the hypothesis 
that depth and rivalry may share overlapping neuronal resources. The ability to voluntarily 
direct attention in accordance with behavioural goals may fit with the conceptualisation of 
voluntary (ie, top-down) attention, which has been distinguished from involuntary attention 
(ie, bottom-up or stimulus-driven) , which usually relies upon the salience of stimulus features 
such as contrast or colour (Yantis 2008). 

In conclusion, the results indicated that the perception of binocular depth and rivalry 
was indeed simultaneous in the plaid patterns. This is one of a few examples of simultaneous 
percepts studied with both psychophysical and physiological methods (eg, Jiang et al 2008). 
The current work also provides a new paradigm in which binocular visual perceptual changes 
are reported concurrently, and do not differ in attentional demand. 
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